Long Short-Term Memory
نویسندگان
چکیده
Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient-based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space and time; its computational complexity per time step and weight is O(1). Our experiments with artificial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with real-time recurrent learning, back propagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artificial long-time-lag tasks that have never been solved by previous recurrent network algorithms.
منابع مشابه
The effect of intrahippocampal microinjection of Naloxone on short –term and long-term memory in adult male rats
Introduction:The hippocampus is one for the major centers of learning and memory. Role of the opioid system has been investigated and on the other hand receptors related to this system such as mu-opioid receptors (MOR) are extended in the hippocampus. In this study the effect of Naloxone administration as a mu opioid receptor antagonist on passive avoidance memory in adult male rats was i...
متن کاملEffects of exercise on spatial memory deficits induced by nucleus basalis magnocellularis lesions
Introduction: Previous studies have shown that exercise enhances cognitive and functional capacities in patients with Alzheimer's disease (AD). In this study, we investigated the effect of long-term (60 days) and short- term (10 days) exercise on the spatial memory deficits in an animal model of AD. Methods: Fifty male rats were divided into 5 groups 1) intact, 2) sham, 3) sham-Alzheimer 4) ...
متن کاملEffect of Short and Long-Term Treatment with Omega-3 Fatty Acids on Scopolamine-Induced Amnesia
Two omega-3 fatty acids including docosahexaenoic acid (DHA) and eicosapentaenoic acid (EPA) are essential for the physiologic function of neuronal cell membrane. Normal function of neuronal cell membrane requires appropriate composition of fatty in its structure. Present study was designed to compare the effect of short-term and long-term pretreatment with omega-3 fatty acids on scopolamine...
متن کاملP7: The Roles of Long-Term Memory on the Organization of the Knowledge for Educators
Modern neuroscientific research help to solve the impotent challenge in curriculum design and teaching for enhancing students’ ability to organize information in a way that makes it efficient in response to an appropriate context such as problem solving and critical thinking via knowing about the mechanism of different type of memories especially long term memory. At first, we should to c...
متن کاملEffect of Short and Long-Term Treatment with Omega-3 Fatty Acids on Scopolamine-Induced Amnesia
Two omega-3 fatty acids including docosahexaenoic acid (DHA) and eicosapentaenoic acid (EPA) are essential for the physiologic function of neuronal cell membrane. Normal function of neuronal cell membrane requires appropriate composition of fatty in its structure. Present study was designed to compare the effect of short-term and long-term pretreatment with omega-3 fatty acids on scopolamine...
متن کاملEffects of fresh, aged and cooked garlic extracts on short- and long-term memory in diabetic rats
Objective: The present study was hypothesized to investigate the beneficial effects of fresh, aged, and cooked garlic extracts on blood glucose and memory of diabetic rats induced by streptozocine (STZ). Material and Methods: Diabetes was induced by an intraperitoneal injection of STZ (60 mg/kg body weight). An oral dose of 1000 mg/kg of each garlic extract was given daily for 4 weeks after dia...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Neural computation
دوره 9 8 شماره
صفحات -
تاریخ انتشار 1997